Calibration

Calibration tells the vision system where the 3D sensor is relative to the robot. Poor calibration is usually the largest source of picking errors, so calibrate carefully and check the result before you deploy.

What calibration does

The 3D sensor reports the pose of each localized object in camera space, which has its origin in the sensor. The robot works in robot space, usually with its origin in the robot base. Calibration finds the calibration matrix: the exact rotation and translation between the two. The Studio applies this matrix to every localized pose, so the robot gets the object pose in its own coordinates.

The calibration matrix converts an object pose from camera space to robot space

Calibration types

The calibration type follows from where the 3D sensor is mounted. You set it in the vision system as Calibration space and sensor position.

Extrinsic calibration (left) and hand-eye calibration (right). In both, the robot stops at 9 poses that are as different as possible, and the Studio scans at each one.

Type	Sensor	Calibration tool	Good to know
Extrinsic Robot space, external fixed sensor position	Fixed in the cell, usually above the bin. It must not move relative to the robot base.	Calibration ball on the flange or gripper	The matrix goes directly to the robot base. The robot can keep moving while the sensor scans.
Hand-eye Robot space, sensor mounted on the robotic flange	On the robot, behind the last joint (flange or gripper)	Marker pattern fixed in the cell	The matrix goes to the robot flange, so the Studio needs the current TCP for every scan. The robot must stand still while scanning, but it can scan from any distance or angle.
Marker space Single sensor position (Locator Studio only)	Usually fixed. On the robot it must always scan from the same pose.	Marker pattern	One scan calibrates the sensor to the marker. Then define a user frame on the robot controller with its origin in the marker origin. Mainly for robots with fewer axes, such as 4-axis robots.
Multi-sensor Static MultiView	Several fixed sensors	Marker pattern	Calibrates each secondary sensor to the primary sensor. See Multi-sensor calibration.

Important

If the sensor, its mount or the robot moves after calibration, the calibration matrix is no longer valid. Calibrate again.

Before you start

• Network – the Vision controller must reach both the 3D sensor and the robot. Configure the sensor interface and the robot interface. See Network setup.

• Vision system – set the Sensor ID and the Calibration space and sensor position. The calibration uses a built-in scanning profile. To use your own, turn on Use custom scanning profile for calibration on the vision system’s Calibration tab.

• Calibration tool – both come with the BPS / LS package.

  • Calibration ball: mount it firmly on the flange or gripper so that most of it is visible to the sensor. You can use your own ball if it is perfectly round and its surface is smooth and not highly reflective.

  • Marker pattern: place it within the robot’s reach and the sensor’s view. It must not move during calibration and must be perfectly flat (check this on custom printed patterns).

• Robot – check the base frame offset, the tool and the robot state input. See Robot setup.

Robot setup

Base frame offset (Bin Picking Studio). Most Fanuc, Kawasaki, Stäubli and Yaskawa robots need a base frame offset so that the Robot State Server reports the TCP in the frame of the virtual robot model. See Base frame offsets and the integration guide for your robot brand.

Coordinate frame (Locator Studio). The camera is calibrated to the frame in which the robot reports its TCP. Choose the frame your application needs. For hand-eye calibration, use the same frame for scans during localization.

Tool. The TCP pose is an input to the calibration. Do not change the selected tool during calibration.

	Extrinsic	Hand-eye
BPS – calibration	Any tool (tool0 or custom)	tool0
BPS – localization	Tool does not matter	Tool does not matter, but the Robot State Server must run
LS – calibration	Any tool (tool0 or custom)	Any tool (tool0 or custom)
LS – localization	Tool does not matter	The same tool as during calibration. Send its TCP in the Scan request.

Robot State Server (Bin Picking Studio). It sends the current robot state, including the TCP pose, to the Vision controller. It is part of every Robot module; set it up with the integration guide for your robot brand. Semi-automatic and automatic calibration need it. Without it, you can still calibrate manually and type in the TCP yourself. During picking it is required only for hand-eye vision systems, but without it there is no live robot visualization.

Locator Studio has no Robot State Server. The operator types in the TCP during manual calibration, and the robot sends it in the Add calibration point request otherwise.

Calibration modes

Mode	Start and save	Add calibration points	Robot pose comes from
Manual	Operator, in the web interface	Operator, with Add calibration point	BPS: Robot State Server, or the operator types it in. LS: the operator types it in.
Semi-automatic	Operator, in the web interface	Robot program, with the Add calibration point request	BPS: Robot State Server. LS: the request.
Automatic	Robot program, with the Start, Save and Stop requests	Robot program, with the Add calibration point request	BPS: Robot State Server. LS: the request.

Manual and semi-automatic calibration are also called calibration performed by the operator. Marker space and multi-sensor calibration are always manual and do not need the robot pose.

The Robot module contains a template program for semi-automatic calibration. Teach your own calibration poses in it before you run it. The start, save and stop requests are already in the template, commented out. Uncomment them to use automatic calibration.

There is also a simulated calibration for testing without a robot.

Calibrate a vision system

These steps are for extrinsic, hand-eye and marker space calibration performed by the operator. Automatic calibration is started by the robot, and its calibration screen is read-only.

1. Start. Open the Calibration step of the solution. Check that the sensor of the vision system is available (click Reload sensors to refresh its status), then click Start calibration.

   

   Calibration step: sensor status, Start calibration and Reload sensors

2. Confirm the parameters.

   • Select ball (extrinsic only): Ping-pong ball (19.95 mm), Snooker ball (26.25 mm), White calibration ball (30.00 mm), or Custom ball with your own radius.

   • Robot state input (Bin Picking Studio, manual calibration only): Robot State Server reads the TCP from the robot. Operator means you type in the TCP from the teach pendant for every point; pick the robot brand in Rotation format so the Studio reads the rotation correctly. If your brand is not in the list, use a brand with the same rotation format, or convert your values to one. In Locator Studio, manual calibration always uses the operator input.

   

   Confirming the calibration parameters

3. Add calibration points. Move the robot to a pose and click Add calibration point (or let the robot program call the request). The Studio scans and looks for the ball or the marker pattern. On the Texture tab, a found ball is marked green; a ball that is partly hidden or confused with a similar object is marked red. Repeat until you have 9 points, using poses that are as different as possible (see Tips).

   For marker space calibration, click Calibrate once instead. Then, without moving the marker, define a user frame on the robot controller with its origin in the marker origin. Watch the direction of the axes.

4. Check the result. After 9 points, the Calibration matrix and the Calibration accuracy appear. Each point also shows its own accuracy. Delete points with a much worse accuracy and add new ones. Up to two points can show N/A: they made the result worse, so the Studio left them out. Replace them with new poses.

   

   Calibration with 9 points, ready to be saved

5. Verify the result on the Verification tab. See Verify the result.

6. Save. Click Finish and save result to store the matrix in the vision system, or Cancel calibration to discard it. Trigger scan takes a new scan without adding a point.

Verify the result

Calibration accuracy is shown in millimeters; lower is better. It depends on the sensor size, the calibration ball, the scanning profile, the lighting and the robot size, so a value of a few millimeters can still be fine. Marker space calibration has no accuracy value.

Bin Picking Studio. The best check is to compare the point cloud with the 3D model of the robot. From the 4th point (extrinsic) or the 6th point (hand-eye), the Verification tab shows the sensor in its calibrated pose together with the point cloud. Move the robot into the sensor’s view and click Trigger scan. If the calibration is good, the point cloud lies on the robot model. This needs a running Robot State Server, so that the model is in the real robot pose.

A good calibration: the point cloud overlaps the robot model

If the overlap is poor, add more points. If the point cloud is completely off, cancel the calibration and start again with different robot poses.

Locator Studio. Check the accuracy value. Also check that the sensor model sits in the right place in the scene: for extrinsic and hand-eye calibration, the scene origin is the robot frame used during calibration (usually the robot base); for marker space calibration, it is the marker origin.

Automatic calibration

Automatic calibration lets the robot program calibrate the vision system without an operator. It repeats an operator calibration using the same robot poses.

Before you can enable it:

• calibrate the vision system manually or semi-automatically first,

• the calibration point dataset must contain at least 9 points, and

• the operator calibration must meet the Minimal allowed accuracy.

Enable it on the vision system’s Calibration tab, under Automatic calibration:

• Minimal allowed accuracy [mm] – if the automatic result is worse, it is not saved.

• Translation threshold (and, for the marker pattern, Rotation threshold) – how far the ball or the pattern may have moved since the operator calibration. Keep the tool firmly mounted.

• Calibration ball type – taken from the operator calibration.

Vision system Calibration tab with the Automatic calibration settings

Use the same calibration poses as in the operator calibration; the dataset on this tab lists them. Changing the robot or the gripper deletes the dataset and turns automatic calibration off.

Simulated calibration

Bin picking solutions only. If there is no robot yet, you can place the sensor by hand to test the solution with recorded scans (path planning in Simulation mode). A solution with a simulated calibration cannot be deployed in Production mode.

Open the Environment step, go to the Vision tab and expand the vision system. Click Edit, set the sensor pose in the form or move the sensor in the 3D view, and click Save to vision system.

Editing the sensor pose for a simulated calibration

Multi-sensor calibration

A Static MultiView vision system merges scans from several fixed sensors, so the sensors must be calibrated to each other.

1. Place the marker pattern where all sensors can see it. It must not move until you finish.

2. On the vision system’s Configuration tab, click Calibrate together below the sensor list.

3. Click Calibrate for each sensor, starting with the primary one, and follow the on-screen instructions. The primary sensor’s matrix is always identity.

4. Optionally, click Fine-tune alignment to refine the scan alignment automatically.

5. Click Save calibration results. The matrices of the secondary sensors are stored on the vision system’s Calibration tab.

Multi-sensor calibration

Calibration sharing

Several vision systems often use the same sensor. To calibrate it only once, turn on Settings > Solution settings > Advanced settings > Share the calibration among vision systems. All vision systems with the same sensor and calibration configuration then share the calibration matrix, the calibration profiles and the automatic calibration settings.

Tips and troubleshooting

• Use robot poses that are as different as possible, with large joint differences.

• Extrinsic: move the ball through the whole bin volume, including the corners and the center.

• Hand-eye: scan the marker pattern from many different robot poses.

• Prefer a larger ball, especially with larger sensors. It gives more points to detect.

Problem	What to do
The whole point cloud is shifted along Z and does not match the robot model.	The robot probably needs a base frame offset. See Robot setup.
The ball is visible in the texture image but not found, although earlier points worked.	The ball may be outside the scanning volume (for example too close to the sensor). Check that it is in the point cloud, for example in PhoXi Control.
Hand-eye (BPS): a warning says the TCP reported by the robot differs from the robot model.	Select tool0 on the robot and check the base frame offset. To compare the two frames, open Axis in the 3D view and show Tool 0 (robot model) and Tool Pose (robot controller).

TCP mismatch warning: the expected and the reported TCP frames do not match